{
 "cells": [
  {
   "cell_type": "raw",
   "id": "547170cb-a3d4-41e8-b555-9ec7f5d8ce61",
   "metadata": {},
   "source": [
    "# 8.7 使用RMSProp优化器训练模型\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c90a18b9-d8e8-431c-856c-35f1e4ed05e1",
   "metadata": {},
   "source": [
    "### 1.任务描述\n",
    "\n",
    "要求：\n",
    "\n",
    "1. 使用鸢尾花数据集构建一层神经网络，实现鸢尾花的分类\n",
    "   \n",
    "2. 使用Python原生代码实现 RMSProp 优化器，对模型参数进行优化，打印迭代损失。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f5b4fc39-cbcf-432a-bf1e-e75e642d4b87",
   "metadata": {},
   "source": [
    "### 2.知识准备\n",
    "\n",
    "见教程。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c1d0295a-4ac4-470a-8263-027a3d69ac2c",
   "metadata": {},
   "source": [
    "### 3.任务分析\n",
    "\n",
    "在参数更新之前，可以通过$\\beta V_{t-1}+(1+\\beta){g_t}^2$计算出更新所需的二阶动量$V_t$，这样在更新时只需对$V_t$执行开方操作即可。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "435c6090-cfda-4f46-a550-22a368e41e4a",
   "metadata": {},
   "source": [
    "### 4.任务实施\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ec75eb6c-5da3-467d-a471-ca3b47242dd6",
   "metadata": {},
   "source": [
    "执行代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2ae9da58-e339-4d22-9f8d-ca255711d89e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0, loss: 0.3352998704130925\n",
      "Epoch 100, loss: 0.027819872595686256\n",
      "Epoch 200, loss: 0.023723917345006153\n",
      "Epoch 300, loss: 0.023181774968693603\n",
      "Epoch 400, loss: 0.022677928198604765\n",
      "Epoch 500, loss: 0.022745898560129447\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from sklearn import datasets\n",
    "import numpy as np\n",
    "# 导入鸢尾花数据集\n",
    "x_data = datasets.load_iris().data\n",
    "y_data = datasets.load_iris().target\n",
    "np.random.seed(116)  \n",
    "np.random.shuffle(x_data)\n",
    "np.random.seed(116)\n",
    "np.random.shuffle(y_data)\n",
    "\n",
    "train_db = tf.data.Dataset.from_tensor_slices((x_data, y_data)).batch(32)\n",
    "# 构建一层神经网络，实现鸢尾花的分类\n",
    "tf.random.set_seed(116)\n",
    "w1 = tf.Variable(tf.random.truncated_normal([4, 3], stddev=0.1, seed=1,dtype=tf.float64))\n",
    "b1 = tf.Variable(tf.random.truncated_normal([3], stddev=0.1, seed=1,dtype=tf.float64))\n",
    "\n",
    "lr = 0.1  \n",
    "epoch = 500  \n",
    "loss_all = 0\n",
    "\n",
    "# 初始化二阶动量\n",
    "v_w, v_b = 0, 0\n",
    "beta = 0.9\n",
    "\n",
    "for epoch in range(0,epoch+1):  \n",
    "    for step, (x_train, y_train) in enumerate(train_db):  \n",
    "        with tf.GradientTape() as tape: \n",
    "            y = tf.matmul(x_train, w1) + b1 \n",
    "            y = tf.nn.softmax(y)\n",
    "            y_ = tf.one_hot(y_train, depth=3,dtype=tf.float64)\n",
    "            loss = tf.reduce_mean(tf.square(y_ - y))\n",
    "        loss_all += loss.numpy()\n",
    "        grads = tape.gradient(loss, [w1, b1])\n",
    "        # 计算二阶动量\n",
    "        v_w = beta * v_w + (1 - beta) * tf.square(grads[0])\n",
    "        v_b = beta * v_b + (1 - beta) * tf.square(grads[1])\n",
    "        w1.assign_sub(lr * grads[0] / tf.sqrt(v_w))\n",
    "        b1.assign_sub(lr * grads[1] / tf.sqrt(v_b))\n",
    "    if epoch % 100==0:\n",
    "        print(\"Epoch {}, loss: {}\".format(epoch, loss_all / 5))\n",
    "    loss_all = 0 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3b3e36e-8418-4caf-af51-d8ac80e2a321",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
